UnREST and Its
Programming Models

It assumes you are comfortable with ordinary web-style programming: HTTP requests and responses, a client talking to a server, a server backed by a database, caches in between. UnREST reuses all of those pieces. What it changes is the default way you think about state — and that one change is what the rest unpacks.

The familiar model: messages

Most networked code moves messages. A sender builds a request, sends it; a receiver handles it and sends a response. If you have written REST endpoints, queued jobs, or socket code, this is the world you know. It works, but the hard parts are well known too: messages arrive late, out of order, twice, or not at all — so you end up dealing with retries, deduplication, ordering, acknowledgements, and timeouts. The correctness of the system depends on messages being delivered and handled correctly.

The UnREST model: shared memory

UnREST changes the default. Instead of sending a message to ask a service to do something, you change a piece of shared memory, and a service reacts to the change. Instead of requesting a value from whoever holds it, you read the value, which is simply there.

Concretely, state lives as named values any node on the network can read or write:

• There is one current value per name. Writing overwrites it in place — no log, no history. If you need to know what changed, compare the new value to the copy you already had.
• The value is location-independent. From your code's point of view there is no fetch and no owner — the value is in shared memory and you read it. (Underneath, the network does route to it; the point is that your code does not have to care where it lives.)
• A read returns the complete current value, never a change-event or a partial update. You read it whenever you need it, even in a tight loop.
• An unchanged read is nearly free. Re-reading a value that hasn't changed collapses to a tiny "nothing changed" marker on the wire. The expensive case in a request/response system — polling something that rarely changes — is the cheap case here.

Messaging is not gone

One clarification, because the slogan is easy to over-read: UnREST does not abolish messages. Request/response is fully supported, and there is an explicit raw mode — a plain HTTP request in, the full HTTP response back — that you can ask for by name when that is genuinely what you want. What changes is what you reach for first. By default you think in shared memory; you drop to explicit request/response only when you deliberately choose to. As Section 4 shows, one of the three services — discovery — does exactly that, because for its job, asking is the right shape.

What it's made of

Underneath, UnREST is an ordinary client/server stack: an object layer your application programs against, a caching proxy and daemon that move data between nodes, and an Apache + database origin that holds the durable state. The one unusual thing lives in the middle — the proxy and daemon share a model of the data, which is what lets an unchanged read cost almost nothing and makes "messages" behave like "memory." Sections 2 and 3 are that machinery; Section 4 walks the three services built on top of it, which sit at increasing depth.

You program against objects

The programming surface is object-oriented. Your application does not assemble packets or parse wire formats. It works with two kinds of object:

• Tuples — the named values in shared memory (covered in Section 3). This is your state.
• Handlers — objects that tell the transport everything it needs to know about a kind of data.

There is a single handler interface — "the ONE UnREST handler interface, no special-casing" — and every concrete handler specialises it. A handler has two faces, and it helps to see them separately.

The three processes (and the fourth tier)

Beneath the object layer are three processes and a database tier. If you picture a caching reverse proxy in front of a web server, you already have the right shape.

The proxy is the client side — a local semantic cache. Whenever your application reads or writes a value, that is, underneath, an ordinary HTTP request to api.php. The proxy intercepts it, uses the matching handler to template it, and computes a hash of the request. The first time it sees a given request it sends the whole thing; the second time it sends only a repeat — the hash by itself — and when the reply comes back as "unchanged," it reconstructs the full current value from the template it already holds.

The daemon is the server side, listening on a fixed port (:9009). A pool of worker threads executes each request against the backend, templates the result with the same handler, compares it to what it returned last time, and replies one of two ways: SAME (a 16-byte identifier, no body) if nothing changed, or DIFF (the identifier plus only the changed fields) if something did.

Apache, api.php, and the database are the origin — an ordinary LAMP tier holding the durable state, which the daemon reaches at 127.0.0.1. The proxy and daemon are a cache pair in front of the real store; they never become the authority. (One service inverts this: a game host serves its origin from the proxy itself rather than from Apache. The shape is still a semantic cache in front of an origin.)

Here is a single read, end to end, the second time it is asked:

Same handler, but not symmetric

The proxy and daemon share the handler, and that shared model is exactly why only a difference has to cross the wire — both ends already hold the structure, so the message shrinks to what changed. But sharing the handler does not mean the two sides do the same thing in both directions. The relationship is asymmetric on purpose. What the proxy says in that shared vocabulary can carry compact or compound meaning the daemon expands and acts on:

• a repeat is just the hash of an earlier request — the daemon looks the hash up, recovers the full question, and routes it to the right executor;
• a diff is only the changed fields of a request — the daemon reconstructs the whole request from the parts it already has;
• a raw request tells the daemon to skip the semantic layer and pass plain HTTP straight through.

The handler gives both sides a common language. The proxy is then free to compose requests in that language that mean more to the daemon than a literal one-to-one HTTP call would — a small token standing for a larger or special operation. Do not assume a request the proxy sends is a mirror of what comes back, or that a one-line request implies a one-line action on the other side.

Why this is "modern" client/server

The tiers are the classic ones, and nothing about Apache or the database is unusual. The difference is the pair in the middle that shares a model of the data. Classic client/server re-sends the entire representation on every request and treats each request as independent. Here, because the proxy and daemon both hold the handler, an unchanged read costs a hash and a changed read costs a diff; and because the server's address is a floating role the mesh resolves, the client neither knows nor cares which physical box answered. Your application is written as though it were reading and writing local objects.

The shared memory is held in a transient database. "Transient" is the important word: it is durable enough to read reliably, but it is not a system of record. Values carry a timestamp and age out if nobody keeps re-asserting them, and the database itself can move from one node to another. You store coordination and live state here, not your permanent records.

Two roles, and why there are two

The transient is not a single fixed server. It exists as two roles, and understanding the split is the most important thing in this section.

Tuples: the values themselves

A single value in the transient is a tuple. You address it with a trio — service / variable / scope — and it carries a JSON value you define. Under the hood it maps onto one row of a simple sensor table (the same api.php schema everything else uses):

Coordination tuples are marked with an SD: prefix on the name (presence, capability, call signaling, and so on). The prefix lets a dead service's leftover values stay recognizable so they can be reaped. Ordinary observed data — sensor readings, link state — has no prefix and is never a reap candidate, so live coordination state and durable data can share the store without being confused.

Scope: whose value it is, and how it expires

Scope is the part people get wrong, so it is worth spelling out. Four helpers cover the common cases:

host_scope()             # "host:<ip>:<pid>"   one running instance; goes away when the process exits
node_scope()             # "host:<ip>"        one durable row per node, outlives any single process
role_scope("mediahost")  # "host:<ip>:<role>" one row per (node, role); two roles never collide
session_scope(call_id)   # "session:<id>"     scoped to a call / session

host_scope includes the process id, so a long-running service gets a value unique to that instance that disappears when it stops. node_scope drops the pid for a value that must be a single row per node and survive restarts (a periodic advertiser, say). role_scope exists because one node can advertise capability under several roles at once — it might be a candidate for both mediahost and databasehost — and since the key does not encode the service, qualifying the scope with the role keeps those rows from overwriting each other.

Writing and reading

In code, it is two calls:

import frognet_tuples as T

# write / refresh this node's presence
T.put("communicator", "presence", T.host_scope(),
      {"name": "Alice", "status": "online"})

# read every node's presence, across all scopes
for r in T.get("communicator", "presence"):
    print(r["addr"], "->", r["value"])

put stamps a timestamp so readers can decide whether a value is still fresh. Its own flag controls cleanup: own=True (the default) means this process owns the value and it is removed when the process exits cleanly — right for live state like presence. own=False is a fire-and-forget refresh that must outlive a short-lived writer and age out by staleness instead; a once-a-minute advertiser uses this, because arming cleanup would delete its value moments later and leave the election with nothing to read.

get(service, variable) returns every instance of that variable across scopes; get_all(service) returns every variable for a service. Passing fresh_s=N drops values older than N seconds — the reader's own freshness decision, made the same way by everyone so they agree.

Floating, re-assertion, and robustness

Because the data host can float, writers re-resolve the host and re-assert their values on a heartbeat, and a background reaper deletes whatever stopped being asserted. The reaper, not cleanup-on-exit, is the real guarantee — a crashing node never gets the chance to say goodbye. One more rule keeps reads safe: a single malformed row is dropped and logged, never allowed to abort the read.

Skip the bad row, note it, and return the rest — one corrupt blob must never empty a whole candidate list.

The three services are the same idea at three depths. Discovery is the shallow end — still moving messages, but so heavily optimized that a stable request is nearly free. The communicator and games are the deep end — they stop thinking in request/response and work in shared memory directly.

4.1 — How a role gets elected

Three of the services run on an elected role — one node the whole network treats as "the one" for a job: the data host, the mediahost, the game engine. Election is not a vote and not a negotiation. It is a deterministic computation each node runs for itself, which is why a split network keeps working on each side and a merged network re-derives the same answer without anyone coordinating. A role is defined by a small handler with two callbacks:

When the network changes, the same computation runs again and a role can land on a different node. Two more callbacks handle the move: advertise re-publishes a node's capability, and hostReset re-asserts a node's own state after the ground shifts under it. Because an elected role keeps no private state — whatever it has acted on is recorded in the shared object it governs — re-running the same logic over unchanged memory does nothing, so a role can float to a new node without dropping the thread.

4.2 — Discovery: messages, highly optimized

Discovery turns a pile of links into a usable network. It works the oldest way: by asking. Each node probes outward from its own interfaces, asks each address who it is, asks each reachable peer what it knows, and from the answers writes itself a routing table. This is still request/response — and that is deliberate. For a job where reachability has to be re-proven constantly and no node's claim can be trusted on its own, asking is the right shape.

What makes it FrogNet rather than plain HTTP is that the asking is nearly free. Discovery asks the same questions over and over, and almost always the answer is the same as last time — so the responses ride the SAME/DIFF compression from Section 2. A peer whose information hasn't moved since you last asked answers in sixteen bytes.

4.3 — The communicator: session memory

The communicator is a deeper use of the model, and the contrast with discovery is the point: where discovery asks questions and acts on the answers, the communicator never asks anyone anything. It works in shared session state — who is online, who is calling whom, who has joined a call. A participant appears by writing its own presence into the shared region, and drops out simply by ceasing to refresh it. The governing role — the mediahost, elected one per local network — reacts to those writes.

4.4 — Games: governed memory

A game is the case that needs a single authority: two players must never see two different boards. So unlike the other two, it has one writer-of-record — the boardgame engine — supplied by the same election that designates every other role, and deliberately kept a different role from the data host so that game authority and data storage need not sit on the same machine.

An AI assistant is a strong collaborator on this kind of system, but it has one specific, predictable failure that will cost you time if you don't watch for it: it quietly re-introduces message-thinking. Request/response, publish/subscribe, send-acknowledge-retry — that is the overwhelming majority of its training, so it is the shape it falls back to whenever it isn't actively held to the model. The symptoms are recognizable:

Two working habits that matter more than anything else

• Make it read the actual source before it describes anything. An AI will describe your code confidently from memory, and it will be wrong about your codebase in ways that sound exactly as confident as when it is right. Require it to open the file and read the function before it tells you what the function does.
• Make it prove, not assert. For any change, ask for a test on the real system that fails on the old code and passes on the new, and that reproduces the actual failure you saw — not a convenient stand-in. And treat a correction from you as a class of mistake to fix everywhere, not a single line to patch.

Used that way — model first, read before describing, prove before believing — an AI is genuinely helpful on UnREST. Used loosely, it will hand you a clean message-passing system that merely calls its messages "memory."

This conceptual layer is meant to be read first and then set aside. The detailed reference covers the wire format, the exact deterministic election rule, the codec internals, and the database schema — each of which is an instance of the model described here.